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Abstract — The explosion of the amount of data stored in cloud 
systems calls for more efficient paradigms for redundancy. While 
replication is widely used to ensure data availability, erasure 
correcting codes provide a much better trade-off between storage 
and availability. Regenerating codes are good candidates for they 
also offer low repair costs in term of network bandwidth. While 
they have been proven optimal, they are difficult to understand 
and parameterize. In this paper we provide an analysis of 
regenerating codes for practitioners to grasp the various trade- 
offs. More specifically we make two contributions: (i) we study 
the impact of the parameters by conducting an analysis at the 
level of the system, rather than at the level of a single device; (ii) 
we compare the computational costs of various implementations 
of codes and highlight the most efficient ones. Our goal is to 
provide system designers with concrete information to help them 
choose the best parameters and design for regenerating codes. 

I. Introduction 

As cloud-based solutions for backup and sharing are being 
offered to users, the amount of storage needed for cloud 
services keeps increasing. In order to lower the costs (e.g., 
hardware, energy) for operating such systems, it is important to 
rely on efficient paradigms. Currently, many systems still rely 
on well-proven replication |1] to provide high availability from 
non-reliable devices. While easy to understand and implement, 
replication is far from being optimal with respect to the 
trade-off between storage and availability Q-Q. Arguably, 
erasure correcting codes can significantly lower the amount 
of storage needed in data-centers. However, with classical 
erasure codes (e.g., Reed-Solomon), repairing after a device 
failure generates many I/Os and requires transferring a large 
amount of information over the network. In large-scale multi- 
site data-centers, the background network traffic due to repairs 
can become prohibitive for large amounts of data stored. 
In this paper, we focus on an attractive alternative, namely 
regenerating codes [5], to lower such network costs. 

Regenerating codes offer the same properties as erasure 
correcting codes with respect to storage and availability. Yet, 
as opposed to erasure correcting codes, regenerating codes sig- 
nificantly lower the network traffic upon repairs. The seminal 
paper |5| on regenerating codes applies network coding to 
storage systems and defines the optimal trade-off between the 
amounts of data stored and transferred. Regenerating codes, 
designed to be as generic as possible, rely on many parameters, 
difficult to grasp in practice where device availability vary 
from a system to another; let alone the fact that many variants 
of regenerating codes exist (e.g., |[6)-|fT0)). 



In order to help choose the right parameters and coding 
scheme, we make the following contributions: 

• We study the influence of the various parameters at the 
system level, depending on storage device availability. We 
show that the optimum at device level does not always 



apply at system level. (Section III i 
• We compare the computational costs of various cod- 
ing schemes for regenerating codes (random codes (6), 
product-matrix codes p), and exact linear codes (7)) to 
the costs of classical erasure correcting codes (Reed- 
Solomon codes). (Section |IV| > 
Previous practical work on regenerating codes focused ei- 
ther only on random codes [ 1 1 1 while we consider several 
other codes; or on a specific system with a specific code fl2) , 
JT3J while we study several codes and give conclusions that 
can be applied broadly. 

II. Model and Background 

We consider a system of n devices connected by a network. 
The system stores files of size A4 that are immutable (i.e., data 
is appended to the system and once written cannot be modified, 
as in p4)). Devices are available with a probability p due 
to temporary disconnections (e.g., reboot, software upgrade, 
short-term network disruption). Devices fail permanently with 
a rate of A failures per month per device (e.g., disk crash, 
device replacement, long-term network disruption). 

When using erasure correcting codes, the file is divided 
into fc blocks and n encoded blocks are produced so that any 
fc encoded blocks allow recovering the file: the file remains 
available as long as at least k devices are available. Hence, the 
resulting system availability is A = Yli=k — P) n ~ % ■ 

Whenever a block is permanently lost, a repair mechanism is 
used to regenerate it. The repair procedure in erasure correct- 
ing codes consists in contacting k live devices, recovering the 
file and encoding it again to produce a new block. Since the 
whole data is read from disks and transferred over the network, 
this procedure has both high I/O costs, which can be reduced 
using specific codes 1 15 1, and high network cost, which can be 
reduced using regenerating codes [5|. In this paper, we focus 
on the latter codes reducing network costs. 

Regenerating codes apply network coding to storage sys- 
tems to offer the best trade-off between network bandwidth 
repair cost 7 and storage cost a. The file is divided into fcA 
sub-blocks. These fcA original sub-blocks are encoded into na 
encoded sub-blocks which are then spread on the n devices 
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Figure 1. Regenerating codes are optimal with respect to storage and repair cost thanks to efficient repair methods. 



(i.e., one group of a sub-blocks is stored on each device) 
so that contacting any k devices allows recovering the file. 
Regenerating codes rely on an additional parameter d, which is 
the number of devices involved in a repair. Whenever a failure 
occurs, if the number of available devices is at least d, the 
following optimal repair method, shown in Figure [lb] can be 
used: (i) the device being repaired fetches f3 sub-block^] from 
each of d available devices (thus 7 = d(3), ( ii) the device stores 
a sub-blocks computed from the dj3 sub-blocks it received. 

Regenerating codes, as explained in [i5j, can be parame- 
terized by the values A and a to minimize either the storage 
(MSR, Minimum Storage Regenerating) or the bandwidth {i.e., 
network repair cost) (MBR, Minimum Bandwidth). This trade- 
off is illustrated on Figure [Ta] 

The seminal paper about regenerating codes relies on ran- 
domized code constructions, namely random linear network 
codes J6). With such codes, the repaired data is not strictly 
equal to the lost data. As a first consequence, such codes 
cannot be maintained in a systematic form. Systematic codes 
suppresses decoding costs and allows direct access to small 
parts of the file since the file can be read directly from 
the k first blocks (or ka first sub-blocks) without decoding. 
As a second consequence, checking the integrity of such 
randomized codes requires to use complex techniques. In order 
to solve these two issues, it has been proposed to rely on 
exact regenerating codes [16|. Various code constructions have 
been proposed with two of the most advanced ones being by 



Suh et al. |7J and by Rashmi et al. |8j. In Section IV we will 
compare the computational costs of these schemes with the 
randomized code constructions [6| and regular Reed-Solomon 
erasure correcting codes. 

Finally, in regenerating codes |5 |, the number d of devices 
to contact during repairs is chosen once for all and cannot be 
changed. Adaptive regenerating codes 1 17 1 relax this constraint 
and allow d to adapt to each repair. However, adaptive regen- 
erating codes currently can only be implemented with random 
linear network codes, which have a high complexity and 
require costly schemes for integrity checking. In the following 
section, adaptive regenerating codes are included for they show 
the best achievable theoretical bound, yet further research is 

'In the rest of the paper, we will focus on scalar codes (/3 = 1) for the sake 
of clarity and for they have lower computational costs. Vector codes (/3 > 1) 
have higher computational costs but are useful for reducing I/Os. 



needed before they can be used in practice. To repair "static" 
regenerating codes when less than d devices are available, k 
available devices must be chosen and the repair must be carried 
by decoding the file before encoding it again leading to a cost 
7' — kA as with regular erasure correcting codes. Hence, 
finding the right value for d is important to avoid using this 
expensive repair by decoding method. 

In the sequel of the paper, we study regenerating codes and 
focus on MSR codes, which offer the same trade-off between 
storage and availability as Reed-Solomon codes. 

III. System level analysis 

In this section, we consider several levels of device avail- 
ability and study regenerating codes by performing an analysis 
at the system level for it matches the real costs observed. At 
the device level, the storage cost is a and the repair cost 
(i.e., network bandwidth) is 7. Hence, at the system level, 
the storage cost is na and the repair cost is T = nXj. 
The cost at the device level is known for decreasing as n 
increases. However, this conclusion does not apply at the 
system level. This section shows some interesting interactions 
between parameters. In the settings shown on the plots in the 
rest of the paper, we consider one file of size M. = 64MB and 
a failure rate of A = 1 failure per month. The repair cost is 
given in MB per month per file stored. This cost scales linearly 
with the number of files, the file size M., and the failure rate A 
(i.e., if 10 files are stored or if A = 10 failures per month, the 
cost is 10 times higher) thus allowing extrapolating results. 

A. How many devices to repair from? 

Let us consider that k and n are chosen to reach a given sys- 
tem availability Regenerating codes require choosing 
an additional parameter d, which is the number of live devices 
contacted during a repair. Theoretical papers suggest that the 
best value for d is d, 



opt 



— 1. However, it turns out that 
this choice is not the best as soon as the device availability is 
p < 1 as we explain in this section. 

Let us define the probability that exactly i devices are 
available as P(X = i) = (1 - p) n ~ l ■ We define 

G = g(X) as the cost of the repair when X devices are 
available. If i > d, we repair using the optimal method (i.e., 
g(i) = ^r^), otherwise we repair by decoding the whole file 
(i.e., g(i) — M). The expected cost at the system level is 
E{G) — Yll=k P{X = When less than k devices are 
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Figure 2. System level repair cost as a function of d and n. The repair cost admits a minimum that is not d 
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available^] the repair is simply delayed. To this end, we plot 
the system repair cost as "A p^^^^ . 

According to theoretical studies j5j, a high value for d helps 
reducing the cost of repairs at the device level. Yet, this also 
increases the probability that less than d devices are available 
thus leading to more frequent repairs by decoding. Hence, 
there should be an optimal value for d. For always available 
devices (p = 1), it appears that d opt = n — 1, as stated in |5j. 



On Figure 2a we consider a system relying on an MSR 
code with n = 32 and k — 16. We consider various device 
availabilities p and plot the system level cost as a function of 
d. We observe that the cost function admits an optimal value 
for d. For low to medium availabilities (p = 0.5 to p = 0.9), 
the optimal value for d is rather low (much lower than re — 1 
value suggested by the literature (5J). For high availability 
p = 0.99, the optimal value is close to re — 1 but is still 
d pt = 29 (d pt =71 — 3). Moreover, choosing d = 31 instead 
of d = 29 when p — 0.99 more than doubles the repair cost. 
Hence, as soon as devices are not highly available (p = 1), 
the designer must choose d according to the device availability 
observed in the system to best leverage regenerating codes. 

The repair cost for erasure correcting codes is the same 
as the cost for regenerating codes with d = k = 16. For 
high device availability (p = 0.99), regenerating codes with 
d = d opt offer a 10 time improvement over erasure correcting 
codes. For medium availability (p — 0.7), with relatively low 



of 80% in rather stable systems (p = 0.99), whereas in highly 
dynamic systems (p = 0.5) the improvement is only 10%. 

B. How to choose the redundancy level? 

As with classical codes, the amount of redundancy must 
be chosen so that the resulting availability P{X > k) = 
S"=fc {T)P 1 ^ ~ p) n ~ l is at least the desired availability A. 
This guides the values of the parameters k and n as studied 
in pj-pl. Yet, with regenerating codes, it might be interesting 
to sacrifice storage efficiency in favor of repair cost by either 
increasing re or relying on MBR codes instead of MSR codes. 
The first approach increases the number of devices redundant 
data is spread on, while the latter increases the amount of 
redundant data on each device without changing n. We study 
MSR codes and set n to get the lowest possible repair cost 
without constraints on the system availability A. 

Let us consider that p m 1 and hence d = re — 1. As initially 
observed by Dimakis et al. (5), very large re, thus allowing very 
large d = n — 1 does not necessarily helps in reducing costs. 
Indeed, the system level repair cost nX"f admits a minimum 
TP 



at n pt = k 



k. When k is large enough, 
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We perform a similar study when p < 1. For each possible 
value n, we choose the corresponding value d that offers the 



d, 



lowest cost. This effect is shown on Figure 2c When applying 
this procedure for multiple values of p and k, we observe that 
k x c(p) where c(p) is a constant that 



dopt ~ 2fc and n opt 



opt 



19, regenerating codes still offer a 3 time improvement depends only on p as shown on Figure [3a| n opt is chosen so 



over erasure correcting codes. 

As explained, the system is rather sensitive to the choice 
of value d. This calls for codes where d can be changed on 
the fly, namely adaptive regenerating codes (fTT), which are 
similar to MSR codes. These codes may seem more practical 
since they can self-adapt to the system, yet they currently lack 
practical code designs. 

Figure [2b] plots repair costs for optimal adaptive regenerat- 
ing codes (ARC) in bold lines alongside regular codes in thin 
lines^] An interesting observation is that adaptive regenerating 
codes, initially designed for highly dynamic systems, perform 
particularly well in rather stable systems. Indeed, when com- 
pared to MSR with optimal d, they provide an improvement 

2 This case remains rare as the system availability P(X > A;) is high. 

3 When using adaptive regenerating codes [17], repairs are performed inde- 
pendently to compare to MSR and MBR on a fair basis (i.e., no coordinated 
multiple repairs). 



opt 

that on average, the number of devices available for repair is 
approximately d op t- These curves also show that for low values 
(p, k), regenerating codes cannot operate efficiently for any 
value (n, d) and erasure-correcting codes (n = k + 1, d = k) 
offer the lowest repair cost (but also a low reliability) for such 
(p, k). However, slightly increasing k is sufficient to leverage 
regenerating codes in spite of low device availabilities p. 

We now study MBR as an alternative to MSR. Indeed, 
increasing n beyond the value needed to ensure the required 
availability for the file, consists in globally increasing the 
redundancy above the minimal level. Using MBR also in- 
creases the redundancy but avoids increasing n (and hence 
the system failure rate nX). To this end, we compare storage 
and repair bandwidth of MSR, MBR and Adaptive codes 
(ARC) for various system unavailabilities (Fig. [3b] > given a 
device availability of p — 0.99. Overall, MBR consumes 
twice the storage space needed for MSR or ARC. MBR 
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Figure 3. The lowest repair cost is for d op t f» 2k and n op t ~ k X c(p) |3a|. Optimal storage and bandwidth |3b|. 



significantly reduce the bandwidth consumption. The system 
designer should choose MBR or MSR depending on resources 
she wishes to save. ARCQ approach and even outperform 
MBR with respect to bandwidth without sacrificing the storage 
efficiency (as efficient as MSR) thus significantly improving 
high availability systems. Yet, currently, no practical code 
designs for ARC are known. This represents a challenging 
research agenda for theoretical work. 

IV. Computational Performance 

Various code designs exist to implement regenerating codes. 
These codes rely on various algorithms and data structures, 
leading to different costs when implemented. In this section, 
we consider both CPU processing and memory costs. We 
compare the following most significant codes, which are Reed- 
Solomon codes for erasure correcting codes, and random linear 
network codes |6j, exact linear codes j7| and product-matrix 
codes [8 1 for MSR regenerating codes. 

A. Memory costs 

We briefly study the costs associated with the data structures 
needed to be loaded in memory for encoding and decoding. 

Reed-Solomon codes are linear codes which rely on an n x k 
matrix containing elements of q bits. For k = 16, n = 32, q = 
16, the resulting encoding matrice is of size 1 Kbytes. 

Linear network codes (random linear network codes [6] or 
exact linear codes [7]) rely on an ka x jiA (approximately 
k 2 x nk for MSR codes). For k = 16, n = 32, q = 16, the 
resulting encoding matrice is of size 256 Kbytes. 

The product-matrix codes (8) rely on a more compact 
scheme and the encoding matrix is of size n x 2a (approxi- 
mately n x 2k for MSR codes). For k = 16, n = 32, q = 16, 
the resulting encoding matrice is only 2 Kbytes. 

Overall, when considering 64 MB data blocks (typical in 
cloud storage systems), the memory requirements for these 
matrices is negligible. Moreover, apart for random linear net- 
work codes, which are non-deterministic codes, the encoding 
matrices, which are the same for all files, are created using a 
deterministic process and need not be stored since they can be 
re-created on the fly when needed. As a consequence, memory 
costs, even if higher for regenerating codes than for erasure 
correcting codes, are only a minor issue and are not relevant 

4 Again, for the comparison to be fair, repairs are performed independently 
without relying on coordinated multiple repairs capability of ARC, which 
would reduce bandwidth consumption even more. 



for choosing one particular code design over another. However, 
as we will explain hereafter, the processing costs are a true 
limitation and vary greatly from one code design to another. 

B. CPU costs 

To compare the processing costs, we implemented in Java 
several MSR codes. All codes implementations have similar 
levels of optimization and rely on a log-table based finite field 
implementation^] We ran these mono-threaded implementa- 
tions on a Pentium E2200. 

We implemented (i) random linear network codes (RL) |6j, 
(ii) exact linear codes (EL) [7^] (Hi) product-matrix codes that 
use a compact representation of codes with efficient encoding 
and decoding algorithms (PM) [8], and (iv) Reed-Solomon 
erasure correcting codes (RS) [18|. 

Regular erasure correcting codes (e.g., Reed-Solomon) in- 
volve linear operations on matrices of size k x k. Such 
operations have a reasonable complexity il(k 2 ). However, 
regenerating codes involve the same linear operations but on 
matrices of size ka x nA (approximately 2k 2 x k 2 when n = 
2k and d = n— 1). Consequently, naive linear implementations 
(EL, RL) (6), 0, (TT) suffer from a complexity of fi(fc 4 ) and 
have high computational costs even for low values of k. It 
can be observed that Product-Matrix codes [8] (PM) rely on 
efficient algorithms departing from classical linear approaches 
thus lowering costs as shown here. 

Figure [4] shows the time needed to process a file of size 
M. = 16MB depending on the parameter k. When considering 



the encoding (Figure 4a i, all regenerating codes (PM, EL, 
RL) perform worse than regular erasure correcting codes (RS). 
However, it is interesting to notice that product-matrix codes 
(PM) clearly outperform regular linear regenerating codes 
codes (EL, RL). The two linear regenerating codes rely on 
the same encoding and decoding algorithm, yet, exact linear 
codes (EL) clearly outperform random linear network codes 
(RL). Indeed, the encoding matrix of exact linear codes (EL) 
is much sparser than the one of random linear network codes 
(RL). Overall, these results are consistent with the asymptotic 
complexities discussed in the previous paragraph. 

'Computational costs using multiplication-table based finite field implemen- 
tations are 30% lower but with such finite field implementation k is limited 
because of constraints on field size. These plots, as well as plots showing that 
all computational costs scale linearly with the file size, were omitted due to 
limited space. 

6 We use a linear implementation of product-matrix codes jij, leading to 
constructions similar to |7j. 
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When considering the repair time (Figure [46) , all regenerat- 
ing codes (PM, EL and RL) exhibit similar costs, with a slight 
advantage for random linear network codes (RL) because their 
randomized repair procedure is simpler. Reed-Solomon codes, 
whose repair rely on a costly decoding followed by an encod- 
ing suffer a cost that increase with k and that is higher than 
the cost of all regenerating codes schemes. As a consequence, 
when repair are frequent, it can be more interesting from a 
computational point of view, to use regenerating codes even 
if encoding and decoding are more costly. 

When considering the decoding time, Reed-Solomon (RS) 
codes have a low cost thanks to a specific decoding algorithm 
and small encoding matrices. Product-matrix codes (PM) 
have a reasonable cost that is much lower than the other 
regenerating codes (EL, RL). Indeed, product-matrix codes 
use a specific decoding algorithm much more efficient than 
the algorithm used for other regenerating codes (EL, RL). 

Figure [5] shows that the encoding time, the repair time and 
the decoding time scale linearly with the file size AL and 
confirms the relatively good performance of product matrix 
codes (PM), and the poor performance of linear codes (EL, 
RL) when compared to classical erasure correcting codes (RS). 

Figure [6] studies the additional cost of relying on systematic 
codes. Systematic codes are interesting because the k first 
devices store non-encoded data. As a result, when performing 
a read on such codes, no decoding is needed {i.e., the decoding 
cost is null). Reed-Solomon codes use a specific encoding 
matrix that encodes directly to a systematic form. As this 
matrix is sparser than the matrix for the non-systematic 
version, the systematic Reed-Solomon (S RS) is faster than 
the non-systematic version (RS). On the contrary exact linear 
regenerating codes (S EL) use a matrix that is costly to create 
and that is denser than the original one leading to higher costs. 



Finally, product matrix codes rely on a pre-coding step that 
is performed before the encoding step. This precoding step 
uses an algorithm similar to the decoding algorithm and as 
such increases the costs: the systematic product matrix codes 
(S PM) are more costly than the non-systematic ones (PM). 

Hence, product-matrix codes are good candidates for replac- 
ing Reed-Solomon codes in practical systems. Their impact on 
memory and CPU remains limited. As shown on Figure [5b] 
their non-systematic form only doubles the encoding costs 
and quadruples the decoding costs when compared to non- 
systematic Reed-Solomon codes. Their systematic form mul- 
tiplies by 7 the encoding costs when compared to systematic 
Reed-Solomon codes. Systematic product-matrix codes should 
be preferred when data is read more frequently than it is 
written, otherwise non-systematic codes are more efficient. 

For MBR codes, which are not the focus of this paper, 
Fractional Repetition Codes flO) , not implemented in this 
benchmark, perform very well since they rely on an efficient 
systematic pre-code (e.g. Reed-Solomon) to produce encoded 
sub-blocks that are then replicated on several devices. Repairs 
and reads are performed using simple transfers without any 
computation. The two other implementations of MBR codes 
are random linear network codes based |6| or product-matrix 
codes based [8|, and since they use the same algorithms at the 
MBR and the MSR point, they will behave similarly to their 
MSR implementation (PM and RL). 

V. Conclusion and Discussion 

We study the impact of various parameters of regenerating 
codes since they can have significant impact at the system 
scale. First, despite common belief, d opt is not necessarily 
cfcpt =n—l and instead should be carefully tuned due 
to its high impact on the repair method efficiency at the 
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system level. Even for high device availability p — 0.99 where 
d = n — 1 seems reasonable, we get a two time improvement 
by choosing d = n — 3. Second, the lowest repair cost for 
MSR codes is obtained with d = 2k and n — k x c(p) with 
c(p) a constant depending on the device availability. Third, 
if sufficient system availability is achieved with a rather low 
n, the designer needing to further lower repair cost should 
favor MBR over using MSR with artificially increased n 
because MBR allows achieving lower repair cost at the system 
scale. Finally, since adaptive regenerating codes theoretically 
outperform both MSR and MBR at the system level when 
looking at the amount of data transferred; providing exact code 
designs for ARC can be a promising theoretical research area 
that would allow implementing ARC in practical systems. 

We also study the computational costs associated with the 
various coding schemes available. We show that the additional 
cost of using regenerating codes (product-matrix codes |8|) 
is reasonable. We have shown that non-systematic product- 
matrix codes outperform other linear code designs and only 
double (resp. quadruple) the cost of encoding (resp. decoding) 
when compared to Reed-Solomon codes. Systematic product- 
matrix codes multiply by seven the cost of encoding when 
compared to systematic Reed-Solomon codes. Hence product- 
matrix codes keep computational costs within reasonable 
values given the savings in term of network bandwidth they 
allow to achieve. As a perspective, to lower systematic codes 
computational costs, it would be interesting to design product- 
matrix like codes that encode directly to a systematic form 
without requiring a pre-processing step. 

Interesting perspectives and ongoing theoretical work for 
applying regenerating codes to practical system concern the 
minimization of I/Os jointly with network repair cost and 
storage cost. Indeed, we focused on regular regenerating codes 
minimizing the network cost. In some systems, minimizing 
the I/O (disk reads) is equally important. Regenerating codes 
are not incompatible with this consideration and this research 
subject is active. Recent work has optimized I/O cost by rely- 
ing either on specific coding schemes (9j, 1 10 1 or considering 
variations of regenerating codes p9| , ]20|. Also, reducing 
I/O is an interesting application for coordinated regenerating 
codes p7) that support repairing multiple devices at once. 
Indeed, if instead of performing t successive repairs, t repairs 
are slightly delayed and performed at once in a coordinated 



way, the I/O for repairs are factored thus reducing the overall 
I/O by a factor t without jeopardizing the optimality with 
respect to network. 
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